Improve performance and reduce latency of Transport.write by attempting to send data immediately if all write buffers are empty by tarasko · Pull Request #619 · MagicStack/uvloop

tarasko · 2024-08-16T23:57:11Z

Current implementation of UVStream.write almost never sends data immediately. Instead the data is stored in the buffer and picked up later by uv_check callback.
This introduces unnecessary latency and CPU overhead.
Despite being a change only in UVStream it also directly benefit _SSLProtocolTransport.write latency since it uses UVStream.write to send ssl frames.

This PR increases RPS rate between echoclient and echoserver by roughly 10%.
echoclient --worker 1 --num 200000
echoserver --uvloop --proto

Because of this change a couple of test had to be tweaked.

get_write_buffer_size cannot reliably return how much data has been already buffered. It doesn't take into account the socket send buffer (which is in the kernel)
writing above limits does not necessarily cause pause_writing because again the data first goes to the socket send buffer

Please note that the current implementation of asyncio does the same thing. It tries to write directly to the socket, and, only if EWOULDBLOCK happens, the data is added to the buffer
https://github.com/python/cpython/blob/c13e7d98fb8581014a225b900b1b88ccbfc28097/Lib/asyncio/selector_events.py#L1065

Apart from that:

I have relaxed fast-path condition further and also removed self._buffer_size > self._high_water in _exec_write. Seems to be working fine.
Added primitive return types to _try_write and _exec_write. Previously there were PyObject_RichCompare calls for values returned by _try_write.
Simplied return value meaning for _try_write. Now it just returns number of bytes written or -1 in case of fatal error.
Removed some signed/unsigned and Py_ssize_t -> int conversions in _exec_write and try_write. Just use Py_ssize_t without conversion when possible

…nd it depends on SO_SNDBUF value

tarasko · 2024-08-20T13:27:20Z

This also increases aiohttp websocket echo performance by 20-25%

tarasko · 2024-08-29T18:48:41Z

Hi @fantix, I saw you recently commited something to uvloop. Do you know if someone could take a look at this PR and maybe give a feedback?

fantix · 2024-09-03T15:22:27Z

Yeah I'll go through all issues/PRs again and include some in the final 0.21 release, it's just all taking some time unfortunately, thanks for the PR and your kind patience!

…is removes unnecessary syscall

tarasko · 2024-10-16T07:28:46Z

Yeah I'll go through all issues/PRs again and include some in the final 0.21 release, it's just all taking some time unfortunately, thanks for the PR and your kind patience!

Hi @fantix, sorry to bother, I know it is open source, voluntary work and you're very busy. But would you have some time to look at this (and other my PRs) any time soon?

Also, 0.21.0 is already released :)

Doch88 · 2024-12-03T12:33:41Z

Can someone from the contributors have a look at this PR? It would be nice to improve the performances!

nevmerzhitsky · 2025-03-07T15:40:08Z

@MagicStack guys please finish this CR.

Doch88 · 2025-07-11T14:44:38Z

Updates on this? @fantix

pedroconceicao · 2025-08-29T08:38:25Z

@fantix are you planning to review and release this soon? 🙏

Elyahou · 2025-11-12T08:04:55Z

@1st1 Is there any news about this PR ?

fantix · 2026-01-09T21:16:40Z

Sorry about the delay! I'm back on this one now.

fantix

Actually instead of replicating the fast-path code, maybe we can just relax the fast-path condition, like:

@@ -424,7 +424,7 @@ cdef class UVStream(UVBaseTransport):
     cdef inline _initiate_write(self):
         if (not self._protocol_paused and
                 (<uv.uv_stream_t*>self._handle).write_queue_size == 0 and
-                self._buffer_size > self._high_water):
+                (self._buffer_size > self._high_water or len(self._buffer) == 1)):
             # Fast-path.  If:
             #   - the protocol isn't yet paused,
             #   - there is no data in libuv buffers for this stream,

(kind of a follow-up of 46dd8f3)

I believe this would have the same performance boost as this PR, yet being compatible with the current code base.

uvloop/handles/stream.pyx

tarasko · 2026-01-14T19:50:52Z

Hi @fantix,
thanks for reviewing this. Could you also take a look at 3 other performance improvements?
#627
#626
#629

The last one gives a decent boost, I would definitely suggest to merge it

Also slightly irrelevant here, but any chance we can figure out why tests are sporadically failing? For my PRs I was just re-running tests 2-3 times until they pass :)

fantix · 2026-01-14T19:56:39Z

Roger, I will get to them soon. I'm spotting at least 2 more flaky tests, I shall fix them first (or skip non-critical ones for now).

tarasko · 2026-01-28T00:49:56Z

Actually instead of replicating the fast-path code, maybe we can just relax the fast-path condition, like:

@@ -424,7 +424,7 @@ cdef class UVStream(UVBaseTransport):
     cdef inline _initiate_write(self):
         if (not self._protocol_paused and
                 (<uv.uv_stream_t*>self._handle).write_queue_size == 0 and
-                self._buffer_size > self._high_water):
+                (self._buffer_size > self._high_water or len(self._buffer) == 1)):
             # Fast-path.  If:
             #   - the protocol isn't yet paused,
             #   - there is no data in libuv buffers for this stream,

(kind of a follow-up of 46dd8f3)

I believe this would have the same performance boost as this PR, yet being compatible with the current code base.

Done.

I have relaxed fast-path condition further and also removed self._buffer_size > self._high_water. Seems to be working fine.
I added primitive return types to _try_write and _exec_write. Previously there were PyObject_RichCompare calls for values returned by _try_write.
Simplied return value meaning for _try_write. Now it just returns number of bytes written or -1 in case of fatal error.

fantix

👍

uvloop/handles/stream.pyx

fantix · 2026-01-29T16:39:12Z

uvloop/handles/stream.pyx

+            if used_buf:
+                PyBuffer_Release(&py_buf)


Good catch!

I added PyBuffer_Release for consistency, but I think that the outer check if blen == 0: is not really necessary.

We already filter out empty data objects twice. First in write(), second in _buffer_write().
We can probably remove the check in write() and leave check only in _buffer_write(). Both write and writelines rely on it.

I can do it, but I'd rather put it into a separate PR

Yeah let's merge this first.

uvloop/handles/stream.pyx

taras added 4 commits August 17, 2024 01:38

Attempt to send data immediately if all write buffers are empty

f4d505c

Fix stale tests

20aa1bb

Don't test get_write_buffer_size because it is difficult to predict a…

fde18ef

…nd it depends on SO_SNDBUF value

Rename written to bytes_written

e360c3a

tarasko and others added 2 commits August 27, 2024 07:17

Merge branch 'master' into optimize_write_latency

b79a840

Fix comment

cfa33ab

tarasko changed the title ~~Reduce Transport.write latency by attempting to send data immediately if all write buffers are empty~~ Improve performance and reduce latency of Transport.write latency by attempting to send data immediately if all write buffers are empty Aug 29, 2024

Merge remote-tracking branch 'origin/master' into optimize_write_latency

96dc94c

tarasko force-pushed the optimize_write_latency branch from be094bc to 96dc94c Compare August 29, 2024 18:31

Merge branch 'master' into optimize_write_latency

94cbc7f

tarasko closed this Sep 5, 2024

Skip fast path in _initiate_write() if we attempted it in write(). Th…

4f66353

…is removes unnecessary syscall

tarasko reopened this Sep 10, 2024

taras added 3 commits September 10, 2024 13:30

Revert unnecessary check

4e96818

Revert unnecessary change

f6f5094

Revert unnecessary change

ede3f38

tarasko mentioned this pull request Oct 16, 2024

uvloop is grouping responses to multiple requests/messages #631

Open

fantix added 2 commits January 9, 2026 16:20

Merge remote-tracking branch 'origin/master' into optimize_write_latency

b0ea7f6

Merge branch 'master' into optimize_write_latency

3dd6c51

fantix requested changes Jan 12, 2026

View reviewed changes

uvloop/handles/stream.pyx Outdated Show resolved Hide resolved

uvloop/handles/stream.pyx Outdated Show resolved Hide resolved

Merge branch 'master' into optimize_write_latency

ae3f8d2

taras added 2 commits January 28, 2026 01:23

Simplify accordint to PR comments, optimize and cleanup

1e19cd5

Add comments

786ee48

More comments

37af342

fantix approved these changes Jan 29, 2026

View reviewed changes

Apply PR comments

b56c375

fantix merged commit 1d9b6e0 into MagicStack:master Jan 30, 2026
16 of 17 checks passed

tarasko deleted the optimize_write_latency branch January 31, 2026 21:45

Conversation

tarasko commented Aug 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tarasko commented Aug 20, 2024

Uh oh!

tarasko commented Aug 29, 2024

Uh oh!

fantix commented Sep 3, 2024

Uh oh!

tarasko commented Oct 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Doch88 commented Dec 3, 2024

Uh oh!

nevmerzhitsky commented Mar 7, 2025

Uh oh!

Doch88 commented Jul 11, 2025

Uh oh!

pedroconceicao commented Aug 29, 2025

Uh oh!

Elyahou commented Nov 12, 2025

Uh oh!

fantix commented Jan 9, 2026

Uh oh!

fantix left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tarasko commented Jan 14, 2026

Uh oh!

fantix commented Jan 14, 2026

Uh oh!

tarasko commented Jan 28, 2026

Uh oh!

fantix left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

fantix Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

tarasko Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

fantix Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

tarasko commented Aug 16, 2024 •

edited

Loading

tarasko commented Oct 16, 2024 •

edited

Loading